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(54) Title: PROGRESSIVE TRANSMISSION OF IMAGES 



(57) Abstract 

A method for the progressive 
transmission of images from a server to 
a workstation comprises encoding the 
image in multi-dimensional format, us- 
ing the orthogonal wavelet transform, 
and storing the encoded image as a dig- 
ital file on the server. A user request- 
ing the image at a workstation sees, 
first of all, a small thumbnail image. 
If further detail is required, a request 
is made to the server which sends a 
"detail" image consisting only of in- 
formation needed to upgrade the low- 
resolution image to a higher-resolution 
image. This approach saves transmit- 
ting across the network information that 
has already been sent in connection 
with the original low-resolution image. 
The system may be of particular use in 
the hospital environment, for the trans- 
mission of medical images. 
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PROGRESSIVE TRANSMISSION OF IMAGES 

The invention relates to progressive transmission cf 

~ images, ^dTrfpiftfcular although net exclusively to the 

5 oroaressive transmission of medical images across a 

computer network from a server to a workstation. 

There is often a need, within the hospital 
. environment, for physicians and surgeons to have rapid 
access to a variety of medical images such as, for 
10 example, x-ray images or CAT scan images. Typically, 

the images may be stored in digital form on a central 
server so that they may be called up as required by the 
user on a local workstation or display client. 

The problem with such an arrangement is that the 
15 transmission cf images, particularly detailed medical 

images, across a network puts a great strain on the 
available bandwidth. In a realistic hospital 

environment, the time taken for transmission of one 
detailed image is likely to b- high because of the 
20 competition for bandwidth arising from other users of the 

system who may be attempting to view different images at 
the same time on other workstations . A further 
difficulty is that the user may wish to see only one cf 
a ser ies of perhaps rather similar images, and it is cf 
25 course extremely inefficient to have to transmit each of 

those images, one by one, in full detail before the user 
is able to choose the image he or she actually requires. 

One method of dealing with this problem is to 
present the user with a set of small "thumbnail" images, 
3 0 each of which is a low- resolution version cf one cf the 

images of the set the user is interested in. The user 
selects the required image using the displayed thumbnai. 
images, the required image then being sent in higher- 
resolution form from the server to the workstation. 
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There are two ways in which the image information 
may be updated from a low-resolution thumbnail image to 
a high-resolution display of the same image. The first 
way is to retransmit the full content of the high- 
resolution image. This represents a wastage of 
bandwidth, since some of the image information is already 
present at the viewing console. The second way in which 
the display may be updated is to transmit only the 
difference between the current image resolution and the 
desired high-resolution image from the server to the 
workstation. In the case of a wide area distributed 
viewing system, the retransmission of full resolution 
images represents a potential problem: for an individual 
image the retransmission of the full resolution image may 
not be significant; for the case in which there are many 
active viewing consoles, and a steady traffic of viewing 
and selection operations, the retransmission of the 
entire image may represent a significant bandwidth 
requirement over the transmission of the detail only. 
0 It is an object of the present invention at least to 
alleviate the problems of the prior art. 

It is a further object to provide an efficient 
progressive transmission system having an improved 
(smaller) bandwidth requirement. 
5 It is a further object to provide a system in which 

progressive transmission may be applied to any selected 
portion of an image that the user wishes to see at 
higher- resolution . 

According to a first aspect of the present invention 
0 there is provided a method of progressively transmitting 

images across a computer network from a server to a 
workstation, the method comprising: 

(a) storing an image on the server as a digital 
file, the file comprising a iow-resciut ion portion 
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representative of a low-resolution version of the 
image, and a detail portion representative cf detail 
in the image not contained within the low-resolution 



version ; 

5 (b) transmitting first information from the low- 

rescluticn portion to the workstation, and 
displaying the low-resolution version of the image 
at the workstation; 

(c) selecting from the displayed image a user- 
10 defined area to be viewed at higher resolution; 

<d) transmitting from the server second information 
from that part of the detail portion" of the file 
which relates to the user-defined area; 

(e) combining at the workstation the second 
15 information and that part of the first information 

which relates to the user-defined area to create a 
higher-resolution version cf the user-defined area; 
and 

(f) displaying at the workstation the higher- 
20 resolution version of the user-defined area. 

The method of the present invention is particularly 
although not exclusively of use in the hospital 
environment, for the transmission of medical images such 
as echocardiographic images. There is often a 

25 requirements in a hospital environment to display a 

selection cf mosaic of images, which may typically be 
image slices of a 3D data set, or frames of an image 
sequence. The present invention allows the individual 
images making up the mosaic to comprise small low- 
30 resolution "thumbnail" images. A user who wishes to see 

-ore detail for cne particular image, simply selects the 
image (for example by clicking on it using the mouse) . 
Further detail is then sent from the server, this further 
detail being combined with the information already held 
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at the workstation to produce a slightly higher 
resolution image. Yet further information may be 
requested from the server if the user needs to see an 
even higher resolution version of the image. In each 
5 case, the only information that needs to be transmitted 

across the network is the detail information, that is the 
difference between the current low-resolution version of 
zhe image already held at the workstation, and the 
required higher-resolution version. 

LC In some embodiments, the information may be sent 

automatically from the server. In other embodiments, a 
specific request may be sent by the workstation to the 
server each time further information is required. This 
request may conveniently be user-generated. 

15 The preferred encoding method is to use the 

orthogonal wavelet transform. This is a transform which 
is discrete in both the time domain and in the scale- 
space (wavelet) domain. The small extent of the 
functions in the spatial domain allows for rapid 

20 convolution. The orthogonal nature of the functions 

used in the preferred embodiment guarantees that the 
minimum amount of information needs to be sent across the 
network . 

The user may select a desired area of the image 
25 which he or she wishes to see in more detail. A desired 

region could for example be picked out on the screen with 
a mouse, preferably by dragging a box over the area of 
interest. If only an area of the image is selected by 
the user, the server sends information only on that 
30 selected area across the network. 

The invention extends to a computer network and/or 
system for operating the method of the present invention. 

The computer network may comprise a local area 
network or a wide area network, connected by wires or by 
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a wire-less link. The term "network" includes any means 
of remote communication between computers, and 
accordingly encompasses (without limitation) 



co mman'ic' a t7i^n~via~~t he t e 1 e p hone "system and/or satellites. 

5 Such means of remote connection may be useful where for 

example the server is in a different hospital, and 
perhaps even in a different country, from the 
workstation . 

The invention may be carried into practice in a 
10 number cf ways and one specific embodiment will now be 

described, by way of example, with reference to the 
accompanying drawings, in which: 

Figure 1 is a block diagram showing, schematically, 
the operation of the discrete wavelet transform; 
15 Figure 2(a) shows the discrete wavelet transform 

decomposition process; 

Figure 2(b) shows the decimation process at 
successive scales ; 

Figure 3 shows the Baubechies 4, 12 and 20 scaling 
20 functions, evaluated at different respective scales; 

Figure 4 illustrates the decimation process in 
graphical form ; and 

Figure 5 shows how the mult i -resolution image may be 
stored . 

25 The preferred embodiment of the present invention is 

a method and/or system for the progressive transmission 
of medical images from a remote file server to a local 
workstation. Each image is stored on the server in a 
multi-resolution format comprising a low- resolution 

3 0 representation which may be called up as a small 

"thumbnail" image on the workstation, and a plurality cf 
"detail" representations. If the user wishes tc see the 
image at lowest resolution, the thumbnail image alone is 
sent. If, having viewed the thumbnail image, the user 
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10 



15 



20 



25 



wishes to receive a higher- resolution representation, the 
first "detail" representation is passed along the network 
to the workstation. At the workstation, the "detail" 
representation is added to the low- resolution image to 



further resolution is required, the user can call upon 
the next "detail" representation, stored on the server, 
which in a like manner is added to the information which 
has previously been passed to the workstation to provide 
an image at the next highest resolution. The process may 
be continued until the user has the image resolution he 
or she requires, or has an image of the maximum 
resolution which is available from the server. 

The first stage in the process is, of course, to 
take an original image and to encode it in digital form 
in a digital data file which forms the muiti -resolution 
representation that is actually stored on the server. In 
order to split the original image up into a low- 
resolution image plus a plurality of "detail" 
representations or images, the preferred embodiment uses 
the orthogonal wavelet transform. 

The orthogonal wavelet transform is a discrete 
transform, similar to the Foulier transform but instead 
of using sin functions wavelet mother functions are used 
instead. Each wavelet mother function has a limited 
extent in wavelet space, which contrasts with the sin 
functions used for Foulier transforms which are of course 
infinite in extent. 

The wavelet transform has the following general 

form : 



provide the required higher-resolution image. 



If yet 
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The discrete wavelet transform, which is used here, 
is the same, except that the values of a and b are 



r e'str i"c t'e"d~x"o^dl"s"cret e values only: In the pre f erred 

5 embodiment, orthogonal wavelets are chosen, for example 

the Daubechies wavelets shown in Figure 3 . 

For any given signal, one can compute a series of 
weights for discrete values of a and b which, when 
multiplied by the basis function and scaled leads back to 
1C the representation of f . This can be done, in principle, 

without losses (other than those arising from rounding 
errors) . The fact that the wavelets are orthogonal 

guarantees the lowest possible error if certain terms are 
thrown away, for example if the user wishes to see an 
15 image of only intermediate resolution. 

The advantage of the present scheme is that the full 
set of coefficients or weights W(a,b) can be arranged 
into a multi-resolution representation of the original 
signal . 

20 As an illustration, consider the one-dimensional 

example shown in Figure 4. If one takes f (n) to be a 
discrete signal to be approximated, one can obtain a 
first approximation by calculating f where f'(n) is 
calculated by determining the mean cf adjacent samples 

25 within f (n) . For example, f (0) is half the sum of f (0) 

and f(l). Likewise, f'(l) is half the sum of f(2) and 
f o) F' (n) may therefore be considered as an 

approximation g(n) to f (n) . One may also define a 
"detail" representation h(n) as the difference between 

30 f (n) and g (n) , 

The process may be repeated, decimating the sample 
each time, thereby creating a plurality of individual 
"detail" representations, each indicative of the 
difference between the current approximation and the last 
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previous approxima t ion . 

To store the image in mult i -resolution format, the 
final approximated signal is kept, along with all of the 
detail signals. 

5 The process may be represented schematically at each 

stage by the block diagram shown in Figure 1. At each 
stage, an approximation signal g(n) and a detail signal 
h(n) is determined from the input signal. The 
approximation signal is then fed back into the output and 
10 a further approximation and a further detail signal 

obtained. At an appropriate stopping point, the final 
very low- resolution g(n) output is retained, along with 
ail of the detail outputs h(n). 

The structure shown in Figure 1 can be used for 
15 multi-resolution orthogonal biorthogonal , or quadrature 

mirror filters (QMF) where each of the above cases is 
specified by the chosen filter coefficients. 

The filters are simply denoted as low pass, g(n) and 
high pass, h(n), to present a general form, but in fact 
20 these filters are related to scaling function, 0(x), and 

wavelet function i£(x) [2] . The decomposition process is 
illustrated in Figure 2. 

Each block has both a low pass filter and a high 
pass filter. The output of each filter is decimated by 
25 a factor of two. The resolution of the output cf the low 

pass filter is also changed because cf losing high 
frequency detail. 

The output of the filter chain, A-* m S, signal 
represents a discrete approximation of signal at the 
30 resolution 2" m . The other output is called the detail 

signal, D z ~ m S , at the resolution 2" . If the original 
signal has N samples, then the total number cf samples in 
the detail and approximation signals is also N. Thus the 
discrete wavelet representation has the same total number 
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cf samples as the original signal. The smoothness of A2 : S 
depends on the shape of the scaling function. In general 
we would wish to have a smooth scaling function, but this 

functrcn should saTtTsf y some condi ticns such as 

5 orthogonality, support width etc. One cf the important 

things is the spatial support ..width) of the scaling 
function which divides the wavelet transform into compact 
into compact and non-compact wavelets. 

In compactly supported wavelets the scaling function 
10 and the wavelet function have mere regularity at the 

expense cf a large number of non-zero coefficients of 
h(n:-, which results in a large support for 0(x). This 
support width increases linearly with their regularity in 
some wavelets such as the so-called Daubechies wavelets 
15 (Daubechies, 1988) . As shown in Figure 3, increasing the 

number of coefficients of h(n), leads to an increase in 
the smoothness of the scaling function. Note that the 
three traces of Figure 3 have been evaluated at 
resolution in inverse proportion to their support, so 
20 that the x axis does not allow comparison of true 

support. Because of the compact support and 

orthogonality the Daubechies wavelets provide a suitable 
basis set. The Mallat basis set is constructed from 
splines in continuous space, then discretized, and hence 
25 is not compact. 

A multi-resolution decomposition is a method of 
representing signals at different scales of 
magnification. The concept cf multi-resolution 

decomposition was initially developed by Mallat [2] to 
5 0 address the problem of characterising image scenes in a 

scale independent manner. The approach cf Mallat is 
essentially a pyramidal decomposition, along the lines of 
Burt and Adelson, but uses values of h(n; and gin) such 
that the decomposition is onto an orthogonal basis. The 
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basic idea is that of successive approximation, together 
with adding detail signal from one approximation to the 
next. Assume that we have a ladder of spaces such that 

... c V 2 ~ V. c - V.. c V_ 2 ... 

5 With the property that if 

f (X) € V. 

then 

f •; x - 2 - k ) 6 V . ; k 6 Z ; and f[2x) e V. . . 

10 If W. is the orthogonal complement of V. in V.. : so we 

have 

vv_. = V; © w. 

Thus, contains the detail signal necessary to go 
15 from V. to V,... If we have an approximation of a signal 

at a resolution corresponding to V. , then a better 
approximation to the signal is obtained by adding the 
detail signal corresponding W : . This detail signal is the 
projection of the signal in W : . 
20 By interpolating the output 

A-~. S 

of the decomposition depleted in Figure (2;, one obtains 
an approximation 



BAD ORIGINAL 



BNSDOCID: <WO 96298 1 8A1 J_> 



WO 96/29818 PCT/GB96/00623 

11 

5 in) 



10 



zz the signal Sin) at a resolution 2 : . The interpolation 
is defined by the following equation: 



S ! .r.) - ^4>(r.-k) A.\- S(k) 



(1) 



wnere 



a;., S 



denotes insertion of a zero between each sample of the 
discrete signal 



A 2 -,S 



This interpolation is a convolution between the 
scaling function <t> and 



a:.-. S 



This particular interpolation represents an example 
in going from a representation cf the signal at one scaie 
2 0 to an approximation of the signal at the scale 
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12 

immediately above it. If we wish to go from a 
representation of the signal at, say, a scale of 2' : to an 
approximation of the signal at some scale 2* : * : , where 
1>1, it is necessary to dilate the scaling function 
5 appropriately. 

The appropriate dilation may be performed as 
follows: if the scaling function of Equation (1) is at 
the resolution l(i,e. 2"*), then it belongs to V : . 

10 Since 



one may write 



$ [2' l n) = 52 h{k) *<^-*> 

k 

15 (2) 

This equation may be generalised to relate the 
scaling function at a scale 2 : to the scaling function at 
a resolution 2 : * : for any j. Therefore, 




For the sake of simplicity, the above examples have 
assumed a one-dimensional signal rather than a two- 
dimensional image, which of course is what we are really 
25 interested in. In dealing with images rather than 

signals, it is necessary to use the corresponding two - 
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dimensional discrete wavelet transform ?DWT) . The two- 
dimensional transform may again be represented by a 
repeated function block where, instead of a pair of 
"outputs - ; — we have four outputs corresponding to the low- 
frequency image component and the high-frequency 
vertical, horizontal and diagonal components. The latter 
three image components correspond to the detail signals. 

For details of the two-dimensional decomposition, 
refer to Mallat [1] . It is possible to generalise 
Eauaticn (3) to the two-dimensional case: 



(4) 

due to the separability of the two-dimensional 
decompos i t ion . 

15 Figure 5 shows, in schematic form, how the 

information may be stored in a data file on the server. 
The lowest resolution image that is expected to be 
required (A) is stored, along with the sequences of 
detail information D : , D 2 and D 3 . If, for example, the 

2 0 lowest resolution image A is an 8x3 image, one will 

automatically obtain from the algorithm three sets of 
detail signals each of which also has size 3x3. The 
next level up is 16 x 16, and so on. To produce an image 
of 16 x 16 resolution, the three detail signals D : are 

25 added into the low-resolution image A. Similarly, to 

create an image of 32 x 32 resolution, the detail images 
3- are added onto the previously-created 16 x 16 image. 

It will cf course be understood that it is not 
necessary for the data file to be stored exactly as it is 
30 shown in Figure 5. All that is required is that the low- 

resolution image can be selected from the file and sent 
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across the network to the workstation followed, if 
requested by the user, by each of the detail images in 
order . 

In looking at medical images, it is often useful to 
5 ca able to view a particular region cf interest in 

greater detail. The infrastructure cf orthogonal, 

r.ult iresoiution image decomposition provides an efficient 
way cf doing this, in terms of bandwidth requirements. 
Consider a region cf interest, such as a heart valve on 

10 a cardiac image. By selecting an approximate region of 

interest on a low resolution, interpolated image, the 
present infrastructure allows one to view that region at 
full resolution without needing to increase the 
resolution outside of the specified region. 

15 The user first selects an area of interest (for 

example using a mouse), and that area is then redrawn on 
the screen in greater detail. For example, if the user 
wishes to view in more detail only the top left-hand 
corner of the thumbnail image A, he or she simply selects 

20 that area and a signal is sent to the server instructing 

it to send further detail appropriate to that area only. 
With the file structure shown in Figure 5, the server 
would send only the detail information contained within 
the top left-hand corners of each of the three D. images. 

25 If the user then wishes to view that enlarged image in 

yet more detail, the server would send only zhe top left- 
hand corners of the D : images . 

In order to refine the detail over a particular 
user-selected area of the image, we need to determine the 

3 0 range of influence of all samples (pixels) within the 

area we are interested in, over all scales cf the 
decomposition process. This range cf influence is 
clearly a function of the width cf the filters used, that 
is the impulse response cf the filters used in the 
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decomposition. Since we are typically using filters of 
very small length, this range of influence is fairly 
localised throughout all space. However, proper 



recon"st'ru"ct''i'of5 can Be carried out only if we can 

5 correctly identify all of the samples, at each scale, 

which influence the desired image within the selected 
area . 

A general implementation involves tracing all of the 
influenced children of a pixel throughout the various 

10 components of the image in the transform space. 

Although tedious, this is quite within the capabilities 
of the skilled man. If orthogonal filters are used, as 
they are in the preferred embodiment, then the range of 
influence simply becomes the direct children (4) of each 

15 pixel. 

This allows great savings in bandwidth. Consider 
the requirement to view a region of microcalcif ication in 
a digital x-ray mammogram at a resolution of 256x256 
pixels. Mammograms typically have full pixel dimensions 

20 of 4096x4096 pixels. In such a situation, immediate 

enhancement of the region requires ail the coefficients 
to go in resolution from, say a 512x512 icon to a full 
resolution region is less than 1% of the total 
coefficients necessary to rebuild the entire image at 

25 4096x4096 resolution. 

The results of our study show that the 
reconstruction of an image from its full wavelet 
decomposition represents by selecting an appropriate set 
of scales represents a valid method of building a multi- 

30 resolution framework for progressive transmission of 

echocardiographic images. We have shown that net only 
does this framework support the progressive updating of 
"thumbnail" images, but also that smooth interpolation cf 
these small images by an appropriately dilated scaling 



BAD ORIGINAL 



BNSDOCID: <WO_96298 1 8A1 J^> 



WO 96/29818 



PCT/GB96/00623 



16 



function lead to low resolution images ac magnifications 
(scales) comparable to the original image. Such blurred 
images may be used, for example, to select with precision 
a particular region of interest, which one wishes tc view 
5 in greater detail (i.e. at full resolution). 

The clear advantage of adopting this approach to the 
clinical viewing cf images in a networked client /server 
environment, such as a teleconferencing or remote 
diagnosis application, is that communications bandwidth 
IC is used optimally. Where a large number cf clients are 

involved, and commercial networks are employed, the 
savings can be significant. Finally, even on local area 
networks using ethernet technology, the cut in overheads 
can represent a significant saving in bandwidth. 
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CLAIMS ; 

1. A method of progressively transmitting an image 
____aG-ro-s-s-a— eomp-u-ter— network-f rom~a"server~t o a^works cation , 
5 the method comprising: 

ia) storing an image on the server as a digital 
file, the file comprising a low- resolution portion 
representative of a low-resolution version of the 
image, and a detail portion representative of detail 
2_q i n -he image not contained within the low-resolution 

version ; 

(b) transmitting first information from the low- 
resolution portion to the workstation, and 
displaying the low-resolution version of the image 

15 at the workstation; 

(c) selecting from the displayed image a user- 
defined area to be viewed at higher resolution; 

(d) transmitting from the server second information 
from that part of the detail portion of the file 

20 which relates to the user-defined area; 

(e) combining at the workstation the second 
information and that part of the first information 
which relates to the user-defined area to create a 
higher-resolution version cf the user-defined area; 

2 5 and 

(f) displaying at the workstation the higher- 
resolution version of the user-defined area. 

2. A method as claimed in Claim 1 in which the detail 
20 portion of the file comprises a plurality of detail sub- 
portions, successive sub-portions containing information 
on successively finer detail within the image. 

3. A method as claimed in Claim 2 in which third 
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information from a first sub-portion is transmitted from 
the server, the third information being combined at the 
workstation with the first information to create a first 
intermediate resolution version of the image. 

4 . A method as claimed in Claim 3 in which fourth 
information from a second sub-portion is transmitted from 
the server, the fourth information being combined at the 
workstation with the first intermediate resolution 
10 version of the image to produce a second intermediate 

resolution version, of higher resolution than the first 
intermediate resolution version. 

5. A method as claimed in Claim 4 in which fifth 
15 information from a third sub-portion is transmitted from 

the server, the fifth information being combined at the 
workstation with the second intermediate resolution 
version of the image to produce a third intermediate 
resolution version, of higher resolution than the second 
20 intermediate resolution version. 

6 . A method as claimed in any one of the preceding 
claims in which the digital file is encoded using a 
transform having a basis function of finite extent. 
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7. A method as claimed in any one of Claims 1 to 5 in 

which the digital file is encoded using the wavelet 
transform . 

9 . A method as claimed in any one of Claims 1 to 5 in 

which the file is encoded using the orthogonal wavelet 
transform. 

9. A method as claimed in Claim S in which the 
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orthogonal wavelet transform uses a Daubechies' wavelet. 

1C. A method as claimed in any cne of the pr eceding 

s— r r~ whi"c h^t ne^' omi^i^i'ng - a he workstation includes 

E interpolation. 

11. A method as claimed in any one of the preceding 
claims including storing a plurality of images on the 
server, and displaying on the workstation a corresponding 

10 plurality of low-resolution versions of the images for a 

user to select. 

12. A method as claimed in any one of the preceding 
claims in which information from the detail portion is 

15 transmitted from the server on receipt of a request from 

the workstation. 

13. A method as claimed in Claim 12 in which the request 
from the workstation is user-selectable. 

20 
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